Image processing apparatus and method of processing image

ABSTRACT

An image processing apparatus includes a first detecting unit configured to detect an object in an image; a determining unit configured to determine a moving direction of the object detected by the first detecting unit; and a second detecting unit configured to perform detection processing of detecting whether the object detected by the first detecting unit is a specific object on the basis of the moving direction of the object determined by the first determining unit.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to an image processing apparatus fordetecting a target object in an image and a method of processing animage.

2. Description of the Related Art

In the past, a specific object, such as a human face or body, wasdetected in an image through pattern matching. For example, a knownimage processing apparatus extracts local patterns by applying amatching pattern to the entire image as the size of the image is changedand calculates local characteristic values. Then, a human body isdetected in the image on the basis of the sum of weighted localcharacteristics (for example, refer to Japanese Patent Laid-Open No.2009-211311). A known image processing apparatus efficiently detects anobject by dividing an image into sections and carrying out processing inaccordance with the priority levels of the sections. The priority levelof each section and each adjoining section in which a target object hasbeen detected is set high (for example, refer to Japanese PatentLaid-Open No. 2008-234169). It has been known that the method of facedetection can be changed depending on the orientation of the camera body(for example, refer to Japanese Patent Laid-Open No. 2010-088093). It isalso known that an object can be tracked through template matching (forexample, refer to Japanese Patent Laid-Open No. 2002-373332).

When a human body is detected by applying a matching pattern to theentire image, the processing load increases as the size of the imageincreases. For example, in a monitoring camera image, the object maymove in any direction and not necessarily move in a direction in whichthe face and body of the object can be easily detected. The detectionaccuracy of human body detection using a specific matching patternchanges depending on the viewing angle and size of the object. Whenhuman body detection is performed by dividing an image into sections andsetting priority levels of the divided sections on the basis of pastdetection results, an object may not be efficiently detected because theobject moves. With monitoring cameras, processing load needs to bereduced to enable real-time processing.

With human body detection using a specific matching pattern, a humanbody may not be detected accurately, regardless of the image quality ofthe object. Even when the method of face detection is changed inaccordance with the orientation of the camera body, the face region maynot be accurately detected.

SUMMARY OF THE INVENTION

To solve the problems described above, the present invention provides animage processing apparatus and a method processing an image that enableefficient detection of an object, regardless of the orientation and sizeof the object.

Accordingly, the present invention provides an image processingapparatus including a first detecting unit configured to detect anobject in an image; a first determining unit configured to determine amoving direction of the object detected by the detecting unit; and asecond detecting unit configured to perform detection processing ofdetecting whether the object detected by the first detecting unit is aspecific object on the basis of the moving direction of the objectdetermined by the first determining unit.

According to the present invention, objects that are more likely to bedetected as a specific object in an image can be detected. Therefore,the efficiency of detecting a specific object in an image is improved.

According to the present invention, an object can be detectedefficiently, regardless of the moving direction of the object.

Further features of the present invention will become apparent from thefollowing description of exemplary embodiments with reference to theattached drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1A illustrates an example configuration of an image processingapparatus according to the present invention.

FIG. 1B illustrates an example configuration of a determining unitaccording to a first embodiment.

FIG. 1C illustrates an example configuration of a determining unitaccording to a second embodiment.

FIG. 2 illustrates the movement of objects in an image.

FIG. 3 illustrates the moving direction angle of an object in an image.

FIG. 4 is an example priority table according to the first embodiment.

FIG. 5 is a flow chart illustrating the operation of an image processingdevice according to the present invention.

FIG. 6 is a flow chart illustrating the process of priority leveldetermination by the image processing apparatus according to the firstembodiment.

FIG. 7A illustrates an example matching pattern for a forward-facingobject according to the second embodiment.

FIG. 7B illustrates an example matching pattern for a lateral-facingobject according to the second embodiment.

FIG. 8 illustrates an example matching pattern table according to thesecond embodiment.

FIG. 9 is a flow chart illustrating the process of matching patternselection.

DESCRIPTION OF THE EMBODIMENTS

Embodiments of the present invention will be described in detail belowwith reference to the accompanying drawings. The configurationsdescribed in the embodiments below are merely examples, and the presentinvention is not limited to the configurations illustrated in thedrawings.

First Embodiment

The configuration of an image processing apparatus according to thisembodiment will be described with reference to FIG. 1A. The componentsillustrated in FIG. 1A may constitute physically independent hardware ormay constitute at least one instance in software having a functionequivalent to the hardware.

An acquisition unit 101 acquires time-sequential images to be processed,which have been captured by a camera. The acquisition unit 101 mayacquire images stored in a server or an external memory.

A first detecting unit 102 detects an object in an image acquired by theacquisition unit 101. The first detecting unit 102 is capable ofdetecting multiple objects. In this embodiment, the information about anobject detected by the first detecting unit 102 contains the position ofthe object in the image, a rectangle circumscribing the object, and thesize of the object. As a method of detecting an object, an inter-framedifferencing technique or a background differencing technique may beused. The method of detecting an object is not limited to an inter-framedifferencing technique or a background differencing technique, so longas the information described above can be acquired.

A tracking unit 103 links an object detected by the first detecting unit102 in an old image acquired by the acquisition unit 101 and an objectdetected by the first detecting unit 102 in a current image acquired bythe acquisition unit 101. Such linking is possible by adding the sameobject ID that has been added to the object detected in the old image tothe object detected in the current image. A new object ID is added to anobject that has newly appeared.

A determining unit 104 determines the content of a detection processdepending on the moving direction the object detected by the firstdetecting unit 102. The content of the detection process includes, forexample, the order of performing detection on each of the multipleobjects detected by the first detecting unit 102 to determine whethereach object is a specific object. The content of the detection processincludes, for example, the type of detection processing (for example,pattern matching or detection based on object size) for determiningwhether each object detected by the first detecting unit 102 is aspecific object. The content of the detection process includes, forexample, detection criteria for performing certain types of detectionprocessing (for example, a matching pattern used in pattern matching, alocal characteristic value used as a reference, etc.). For example, whenperforming detection processing by pattern matching, the contentcontains information about which matching pattern is to be used as thedetection reference. When performing detection processing using localcharacteristic values, the content contains information about whichlocal characteristic value is to be used as a detection reference.

FIG. 1B illustrates the configuration of the determining unit 104according to this embodiment. A first determining subunit 110 determinesthe moving directions of the objects detected by the first detectingunit 102.

A second determining subunit 111 determines the priority levels used fordetermining whether each object is a specific object on the basis of themoving directions of the objects determined by the first determiningsubunit 110.

A third determining subunit 112 determines which object among theobjects detected by the first detecting unit 102 is to receive detectionprocessing on the basis the characteristic values of the objectsdetected by the first detecting unit 102. That is, the third determiningsubunit 112 determines whether detection processing by the seconddetection unit 105, which is described below, is to be performed on eachobject detected by the first detecting unit 102 on the basis of thecharacteristic value of the corresponding object.

The process of determining priority levels by the determining unit 104will be described with reference to FIGS. 2 and 3. FIG. 2 illustrates anexample of the moving directions of objects in an nth frame. The objectrepresented by an object ID 12 has moved from position 201 in the(n−1)th frame to position 202 in the nth frame; thus the movement of theobject ID 12 can be represented as a position vector 203. The objectrepresented by an object ID 6 has moved from position 204 in the (n−1)thframe to position 205 in the nth frame; thus the movement of the objectID 6 can be represented as a position vector 206. In this embodiment,the moving direction angle is determined on the basis of a positionvector based on two frames. Instead, however, the moving direction anglemay be determined on the basis of the scalar product of position vectorsof multiple frames. When the first detecting unit 102 detects an objectby a movement vector, the moving direction of the object may bedetermined from the movement vector. In this way, the first determiningsubunit 110 of the determining unit 104 determines the moving directionsof the objects detected by the first detecting unit 102.

FIG. 3 illustrates an example of a moving direction angle of an object.In this drawing, the rightward direction is the forward direction alongthe x axis, and the downward direction is the forward direction alongthe y axis. A moving direction angle 302 of a position vector 301 isdetermined by setting the forward direction along the y axis as 0°. InFIG. 3, α represents the moving direction angle 302.

In this embodiment, the second determining subunit 111 of thedetermining unit 104 determines an evaluation value from one ofExpressions 1 to 4 on the basis of the moving direction angle 302. Thedetermining unit 104 assigns higher priority levels to smallerevaluation values. Expression 1 is used when the moving direction angleα is not smaller than 0° and smaller than 90°; Expression 2 is used whenthe moving direction angle α is not smaller than 90° and smaller than180°; Expression 3 is used when the moving direction angle α is notsmaller than 180° and smaller 270°; and Expression 4 is used when themoving direction angle α is not smaller than 270° and smaller than 360°;When objects have the same evaluation value, the object having thesmaller object ID is assigned with a higher priority level.Val dir−α  (1)Val dir=|α−180|  (2)Val dir=α−180  (3)Val dir=|α−360|  (4)

In this way, the second determining subunit 111 of the determining unit104 sets higher priority levels for objects moving vertically in theimage and lower priority levels for objects moving horizontally in theimage. The method of assigning priority levels is not limited thereto.For example, if larger evaluation values are assigned with higherpriority levels, objects moving horizontally in the image can beassigned with higher priority levels than objects moving vertically inthe image. Instead, priority levels may be assigned in accordance withthe moving direction angle 302, without using Expressions 1 to 4. Whenthe first detecting unit 102 detects multiple objects, the seconddetermining subunit 111 of the determining unit 104 assigns prioritylevels to the objects on the basis of the moving directions of theobjects. When the first detecting unit 102 detects multiple objects, thedetermining unit 104 according to this embodiment assigns prioritylevels to the objects so as to determine whether the characteristicvalues of the objects satisfy a predetermined condition in a descendingorder of priority level. That is, in this embodiment, the determiningunit 104 assigns priority levels to regions on which human bodydetection is to be performed on the basis of the size of the objectsdetected by the first detecting unit 102 and the tracking results of thetracking unit 103.

Then, the third determining subunit 112 of the determining unit 104determines the size of the objects in a descending order of prioritylevel. When the size of an object is smaller than a predeterminedthreshold, the third determining subunit 112 sets a skip flag indicatingthat detection processing, which is described below, is not to beperformed. The threshold of object size is a value set in advance by aninstruction from a user. FIG. 4 illustrates example results of settingpriority levels and skip flags.

Information illustrated in FIG. 4 is stored in a storage unit 108. Askip flag is set on the basis of the size of an object detected by thefirst detecting unit 102; instead, a skip flag may be set on the basisof some other value. A skip flag may be set on the basis of anyattribute that can be used to sort the objects, such as movingdirection, object height, object width, change of object area, aspectratio of object shape (ratio of height to width of object), or movingspeed of object. In this way, the third determining subunit 112determines, on the basis of the characteristic values of the detectedobjects, whether detection processing by the second detection unit 105is to be performed on the detected objects in an order corresponding tothe moving directions determined by the first determining subunit 110.

By not performing detection processing on objects having skip flags, thedetection probability of detecting a target object among the objectsdetected by the first detecting unit 102 can be increased. The amount ofprocessing required for detecting a target object among the objectsdetected by the first detecting unit 102 can be decreased. However,detection processing using skip flags is not an essential step in thepresent invention. After determining priority of the objects detected bythe first detecting unit 102, the detection processing step describedbelow may be performed without setting skip flags.

The second detection unit 105 detects a human body by performing patternmatching on regions corresponding to the objects detected by the firstdetecting unit 102. The method of detection is not limited to patternmatching. The second detection unit 105 according to this embodimentperforms human body detection on the objects in order from the objectwith the highest priority and not having priority and skip flags set bythe determining unit 104. In this way, the second detection unit 105determines whether to perform detection processing on the objectsdetected by the first detecting unit 102 to detect a specific object onthe basis of the skip flags set by the determining unit 104. When skipflags are not set, the second detection unit 105 performs human bodydetection on the objects in a descending order of priority level.

In this way, the second detection unit 105 detects a specific objectamong the objects detected by the first detecting unit 102 in an ordercorresponding to the priority levels determined by the determining unit104. The priority levels are assigned on the basis of the movingdirections of the objects. For example, when the second detection unit105 is to detect a human body in an acquired image, human body detectionis performed on an object that is approaching the camera used foracquiring the image with priority over objects moving in otherdirections. Objects that are moving away from the camera may also beassigned with the same priority level as that of approaching objects.Whether an object is approaching the camera is determined on the basisof the moving direction in the image, i.e., the angle of the movingdirection to a vertical axis in the image. Instead, the detection ordermay be directly set on the basis of such angles, without assigningpriority levels.

For example, when a specific human face image is used as a pattern forpattern matching, by placing priority on an object that is approachingthe camera, as described above, pattern matching can be performed on aface facing the camera with priority over objects facing otherdirections. In this way, by performing detection on images thatintensely show the characteristic of a target object, the accuracy ofdetecting the target object can be increased. Since detection isperformed with priority on objects in a descending order of probabilityof detecting a target object, detection can be performed efficiency.

In this embodiment, the detection target is a human body, but thedetection target is not limited thereto. The detection target may be anyspecific object, such as a human face, an automobile, an animal, or asolid body. The second detection unit may be configured to detectvarious different specific objects, enabling multiple detectionprocesses to be carried out simultaneously.

A desired process may be selected among the multiple detection processesin accordance with the object attribute used for setting skip flags,such as the size of the object or the aspect ratio, to performdetection. For example, a human-body detection unit and an automobiledetection unit may be provided as the second detection unit 105. A usermay set, in advance, a threshold defining the minimum size of anautomobile shown in an image. When the size of the object detected bythe first detecting unit 102 is smaller than the minimum automobilesize, processing is not carried out by the automobile detection unit butcarried out only by the human-body detection unit. In contrast, when theobject size is larger than the threshold, processing is carried out bythe automobile detection unit. When the object size is larger than thethreshold, it is possible that a group of people has been detected as asingle object; therefore, processing is also carried out by thehuman-body detection unit. In this way, the second detection unit 105selects one of the specific objects as a target object to be detected inaccordance with the characteristic values of an object detected by thefirst detecting unit 102 and performs detection. Accordingly, aplurality of detection targets can be efficiently detected.

A terminating unit 106 determines the termination of a human-bodydetection process performed by the second detection unit 105 on thebasis of the elapsed processing time. The threshold of elapsedprocessing time for determining termination is a value set in advance inaccordance with an instruction from a user. Determination of thetermination of a detection process is not limited to that based on theamount of time that has elapsed from the start of the detection process.Termination of a detection process may be determined on the basis of,for example, the number of human bodies detected by the second detectionunit 105 and the number of times the second detection unit 105 hasperformed a human-body detection process. An output unit 107 outputs, toan external unit, the detection result of the first detecting unit 102,the tracking state of the tracking unit 103, and the detection result ofthe second detection unit 105.

FIGS. 5 and 6 are flow charts illustrating the operation of the imageprocessing apparatus according to this embodiment. For an imageprocessing apparatus with a built-in processor and memory, the flowsillustrated in FIGS. 5 and 6 represent programs for the processor toexecute the steps in FIGS. 5 and 6. The processor built-in the imageprocessing unit is a computer and executes programs read out from thememory built in the image processing apparatus. The memory built in theimage processing apparatus is a recording medium holding the programs ina form readable by the processor.

First, the acquisition unit 101 determines whether or not to continueacquiring images (f1). The decision in Step f1 can be made on the basisof a user instruction input via an input unit (not shown). Instead,image acquisition may be ended when a target object is detected in animage. The method of making a decision is not limited thereto. If imageacquisition is to be continued, the acquisition unit 101 acquires animage from an external unit (f2).

Next, the first detecting unit 102 detects an object in the imageacquired by the acquisition unit 101 (f3) In this embodiment, whendetecting an object, the first detecting unit 102 acquires informationabout the position of the object on the image, the rectanglecircumscribing the object, and the size of the object. As a method ofdetecting an object, an inter-frame differencing technique or abackground differencing technique may be used. The method of detectingan object is not limited to an inter-frame differencing technique or abackground differencing technique, so long as the information describedabove can be acquired.

Subsequently, the detection result is determined (f4). That is, it isdetermined whether the first detecting unit 102 has detected an objectin the image acquired by the acquisition unit 101. If the firstdetecting unit 102 has not detected an object (NO in Step f4), theprocess returns to Step f1.

If an object is detected (YES in Step f4), the tracking unit 103 tracksthe object (f5). In the tracking process, the tracking unit 103 tracksthe object detected by the first detecting unit 102. The tracking unit103 does not have to track all of the objects detected by the firstdetecting unit 102. For example, whether tracking is to be performed maybe determined on the basis of information about the object detected bythe first detecting unit 102 (for example, the position of the object inthe image, the rectangle circumscribing the object, and the size of theobject). For example, when the size of a detected object is smaller thanthe threshold, tracking may not be performed on the detected object. Notonly the object size but also other sets of information detected by thefirst detecting unit 102 may be compared to thresholds to determinewhether an object should be tracked. In this way, the amount ofprocessing required for tracking multiple detected objects by thedetermining unit 104 can be reduced.

Then, the determining unit 104 determines the content of the detectionprocess (f6). The determining unit 104 determines the content of thedetection process in accordance with the moving directions of theobjects detected by the first detecting unit 102. The content of thedetection process includes, for example, the order of performingdetection processing on the multiple objects detected by the firstdetecting unit 102. The content of the detection process includes, forexample, the type of detection processing for detecting a specific inthe objects detected by the first detecting unit 102 (for example,detection processing based on pattern matching, object size, etc.). Thecontent of detection process includes, for example, which detectioncriterion for performing a specific type of detection processing is tobe selected for performing detection processing (for example, a matchingpattern used in pattern matching, or local characteristic values used asreferences). In this embodiment, the determining unit 104 determines thepriority levels of objects on which detection processing is performed.

Next, it is determined whether the detection processing is to becontinued or ended on the basis of a decision made by the terminatingunit 106 (f7). For example, when the first detecting unit 102 detectsmultiple objects in an image, the terminating unit 106 determineswhether to perform detection processing on a second object, afterdetection processing is performed on a first object. The terminatingunit 106 determines whether to continue or end detection processing onthe basis of, for example, the number of human bodies detected by thesecond detection unit 105 or the number of times the second detectionunit 105 has performed human-body detection. Instead, the terminatingunit 106 may determine whether to continue or end detection processingon the basis of a user instruction input via an input unit (not shown).Instead, detection processing may be ended when a target image isdetected. The method of making the decision is not limited thereto. Whendetection processing is to be continued (YES in Step f7), the seconddetection unit 105 performs human-body detection on object areas in adescending order of priority. When detection processing is terminated(NO in Step f8), the process returns to Step f1.

Next, details of priority order determination in Step f6 according tothis embodiment will be described in detail with reference to the flowchart in FIG. 6. The determining unit 104 determines the movingdirection angle of an objects detected by the first detecting unit 102(f60).

Next, the determining unit 104 determines the moving direction angle 302(angle α) (f61). In this embodiment, the determining unit 104 determinesthe angle range contains the angle α: not smaller than 0° and smallerthan 90°; not smaller than 90° and smaller than 180°; not smaller than180° and smaller than 270°; and not smaller than 270° and smaller than360°.

When the angle α is not smaller than 0° and smaller than 90°, thedetermining unit 104 uses the angle α and Expression 1 to determine anevaluation value (f64). When the angle α is not smaller than 90° andsmaller than 180°, the determining unit 104 uses the angle α andExpression 2 to determine the evaluation value (f65). When the angle αis not smaller than 180° and smaller than 270°, the determining unit 104uses the angle α and Expression 3 to determine the evaluation value(f66). When the angle α is not smaller than 270° and smaller than 360°,the determining unit 104 uses the angle α and Expression 4 to determinethe evaluation value (f67).

After determining the evaluation value in this way, the determining unit104 determines the priority order for detection processing for eachdetected object on the basis of the determined evaluation value (f68).For the same evaluation values, the determining unit 104 sets an objecthaving a smaller object ID with higher priority.

The angle ranges listed above are merely examples and are not limitedthereto. In Step f61 in this embodiment, the process follows fourdifferent flows; instead, there may be more or less than four flows inwhich different detection methods are used. Then, the priority levelsdetermined by the determining unit 104 are stored in the storage unit108 (f69). In this way, the determining unit 104 according to thisembodiment performs priority order determination in Step f6.

With the configuration described above, objects that are more likely tobe detected as a specific object are detected. Accordingly, theefficiency of detecting a specific object in an image is improved.

Second Embodiment

In the second embodiment, the determining unit 104 changes type ofdetection processing in accordance with the moving directions ofobjects.

In this embodiment, the determining unit 104 selects a matching patternto be used in the human-body detection process, which is describedbelow, on the basis of the size of the object detected by the firstdetecting unit 102 and the tracking result of the tracking unit 103. Inthis embodiment, the second detection unit 105 can perform detectionprocessing by selecting a matching pattern that is more likely to detectan object in accordance with the orientation of the object, which ispresumed from the moving direction of the object detected by the firstdetecting unit 102. The configuration of the other components is thesame as that described in the first embodiment with reference to FIG.1A; therefore, descriptions thereof will not be repeated.

The configuration of the determining unit 104 according to thisembodiment will be described with reference to FIG. 1C. The determiningunit 104 according to this embodiment includes a matching-patternselecting subunit 121, instead of the second determining subunit 111according to the first embodiment. The matching-pattern selectingsubunit 121 selects a matching pattern for matching an object detectedby the first detecting unit 102 in accordance with the moving directionof the corresponding object.

Example matching patterns are illustrated in FIGS. 7A and 7B. FIG. 7Aillustrates example matching patterns for detecting a forward-facinghuman body. A matching pattern 701 is, for example, a luminance patternof 30×80 pixels. The rectangles included the matching pattern 701 arelocal patterns. A local pattern 702 is used for extracting the edgestrength in the horizontal direction of the head area. Local patterns703 and 704 are used for extracting the edge strength of the shoulderline. Local patterns 705 and 706 are used for extracting the edgestrength in the vertical direction of the arms. Local patterns 707 and708 are used for extracting the edge strength in the vertical directionof the legs. The local patterns described above are merely examples andare not limited to the four areas illustrated in FIG. 7A. An examplematching pattern for detecting a lateral-facing human body isillustrated in FIG. 7B. The matching patterns in FIGS. 7A and 7B differin the positions and sizes of the local patterns and the calculationmethod of local characteristic values, which is described below. Thematching patterns are not limited to forward-facing and lateral-facingobjects; matching patterns corresponding to objects positioned atdifferent angles, such as an oblique angle or a top angle, may also beused. The matching patterns are not limited to those corresponding tothe entire body; matching patterns corresponding to the upper body, theface, or the feet may be used. The matching patterns are not limited tothose corresponding to a human body; when detecting an object other thana human body, a matching pattern corresponding to a specific object,such as a vehicle, a train, or an airplane, may be used.

Next, a method of determining a matching pattern to be used inhuman-body detection performed by the determining unit 104 will bedescribed. The first matching subunit 110 of the determining unit 104 isthe same as that according to the first embodiment illustrated in FIGS.2 and 3 and determines the moving direction of an object detected by thefirst detecting unit 102. The matching-pattern selecting subunit 121selects a matching pattern to be used in human-body detection inaccordance with a moving direction angle α.

In this embodiment, the matching-pattern selecting subunit 121 selects aforward-facing matching pattern for detection processing when the movingdirection angle α is: not smaller than 0° and smaller than 45°; notsmaller than 135° and smaller than 225°; or not smaller than 315° andsmaller than 360°. The matching-pattern selecting subunit 121 selects alateral-facing matching pattern for detection processing when the movingdirection angle α is: not smaller than 45° and smaller than 135°; or notsmaller than 225° and smaller than 315°.

In this way, the determining unit 104 selects a forward-facing matchingpattern for detection processing when an object detected by the firstdetecting unit 102 moves in the vertical direction in the image. Incontrast, the determining unit 104 selects a lateral-facing matchingpattern for detection processing when an object detected by the firstdetecting unit 102 moves in the horizontal direction in the image. Thus,the determining unit 104 can appropriately select a matching pattern fordetection processing in accordance with the movement of the detectedobject.

The angle ranges of the moving direction angle α corresponding todifferent flows are not limited to those mentioned above and can be setappropriately. When the moving direction angle α is a predeterminedangle or is within a predetermined angle range, the determining unit 104decides not to perform detection. In this embodiment, the movingdirection angle is determined on the basis of a position vector based ontwo frames. Instead, however, the moving direction angle may bedetermined on the basis of the scalar product of the position vectorsbased on multiple frames. If the first detecting unit 102 detects anobject by a movement vector, the moving direction of the object may bedetermined on the basis of the movement vector.

FIG. 8 illustrates an example of matching patterns for human-bodydetection selected by the determining unit 104. The informationillustrated in FIG. 8 is stored in the storage unit 108. The informationstored in the storage unit 108 is not limited to that illustrated inFIG. 8 and may be any form of information that represents the linkbetween the objects detected by the first detecting unit 102 and thematching patterns. The objects detected by the first detecting unit 102and the matching patterns do not necessarily have to have a one-to-onecorrespondence but may have a one-to-many correspondence. By linkingmultiple matching patterns to a specific moving direction angle 302, anobject detected by the first detecting unit 102 and multiple matchingpatterns can be linked.

The second detection unit 105 uses a matching pattern selected by thedetermining unit 104 and performs human-body detection on an areadetected by the first detecting unit 102. The second detection unit 105extracts the edge strength of locations detected by the first detectingunit 102 in accordance with the local patterns of the matching patternselected by the determining unit 104. Then, the determining unit 104calculates the scalar products of extracted edge strengths and specificedges in the local patterns to determine local characteristic values u.Next, the second detection unit 105 determines the sum of the localcharacteristic values weighted with w and detects a human body bycomparing the sum with a predetermined threshold th. The method ofcalculating local characteristic values is not limited to calculatingthe scalar product; other methods, such as predetermined linerdetermination analysis, may be used. In this way, the second detectionunit 105 according to this embodiment selects the matching pattern usedfor matching an object detected by the first detecting unit 102 inaccordance with the object moving direction to perform detectionprocessing.

In this embodiment, a human body is a detection target, but thedetection target is not limited thereto. For example, the detectiontarget may be an automobile, an animal, etc. The second detection unit105 may detect various different specific objects. The method ofdetecting a specific object in an image by the second detection unit 105is not limited to pattern matching. For example, an object may bepresumed as a human body on the basis of aspect ratio, size, and/orshape of the object. When performing human-body detection, if a personmoves horizontally in the image, characteristic parts, such as the face,are not captured in the image; therefore, the accuracy of human-bodydetection is low if pattern matching is applied. Thus, pattern matchingmay be applied for detection when the detected object moves verticallyin the image, and aspect ratio, size, and/or shape of the object may beused for detection when the object moves horizontally in the image. Inthis way, the second detection unit 105 can perform detection processingby selecting a matching pattern that is more likely to detect an objectin accordance with the orientation of the object, which is presumed fromthe moving direction of the object detected by the first detecting unit102.

Similar to the first embodiment, when the third determining subunit 112of the determining unit 104 does not perform image processing of theobject, the second detection unit 105 can be set not to performdetection processing.

In this embodiment, a case in which the determining unit 104 changes thematching pattern in accordance with the moving direction angle has beendescribed. Instead, however, the determining unit 104 or the seconddetection unit 105 may change the local characteristic values inaccordance with the moving direction. For example, the weight added to alocal characteristic may be changed in accordance with the movingdirection. When detecting a human image in a video movie, if thedetected object is moving vertically in the image, the weights added tothe local characteristic values of the upper body, such as the face andshoulders, are increased because it is presumed that, if the object ishuman, a front view of the object is being captured. In contrast, if thedetected object is moving horizontally in the image, the weights addedto the local characteristic values of the lower body, such as the feet,are increased because it can be presumed that, if the object is human, alateral view of the object is being captured.

As described above, the second detection unit 105 according to thisembodiment changes the type of detection processing in accordance withthe moving directions of multiple objects determined by the firstdetermining subunit 110. In this way, a pattern preferable for detectinga human image in accordance with the orientation of an object can begenerated. Thus, the probability of detecting a desired object can beincreased.

Next, the operation of the image processing apparatus according to thisembodiment will be described with reference FIGS. 5 to 9. First, theoperation of the image processing apparatus according to this embodimentwill be described with reference to FIG. 5.

Steps f1 to f5 in FIG. 5 are the same as those in the first embodiment;therefore, descriptions thereof will not be repeated. In thisembodiment, in Step f6 in FIG. 5, the tracking unit 103 links objectsdetected by the first detecting unit 102 in multiple images acquired bythe acquisition unit 101. Then, the determining unit 104 selects amatching pattern to be used for detection (f6). The determining unit 104changes the matching pattern to be used for detection in accordance withthe moving direction of the object. Instead, the determining unit 104may change the weights added to the local characteristic values of thematching pattern in accordance with the moving direction of the detectedobject. The operation in Step f6 will be described in detail below withreference to FIG. 7. When the determining unit 104 selects a matchingpattern, the image processing apparatus according to this embodimentcarries out Steps f7 and f8. Since Steps f7 and f8 are the same as thosein the first embodiment, descriptions thereof will not be repeated.

Next, the operation of selecting a matching pattern in Step f6 of thisembodiment will be described in detail with reference to the flow chartin FIG. 9. The determining unit 104, first, determines the movingdirection angle of an object detected by the first detecting unit 102(f90). Next, the determining unit 104 determines whether the movingdirection angle 302 is within a predetermined angle range (f91). In thisembodiment, the angle α, which is illustrated in FIG. 4, is determinedto be in an angle range of: not smaller than 0° and smaller than 45°;not smaller than 135° and smaller than 225°; or not smaller than 315°and smaller than 360°. When the moving direction angle 302 is within apredetermined angle range (YES in Step f91), the determining unit 104selects a forward-facing matching pattern for detection (f93). When themoving direction angle 302 is not within a predetermined angle range (NOin Step f91), the determining unit 104 selects a lateral-facing matchingpattern for detection (f94). The weights of the local characteristicvalues of the matching pattern used for detection may be changed inaccordance with the determination result of Step f91. For example, theweights of the local characteristic values of the upper body may beincreased when the moving direction angle 302 is within a predeterminedangle range, and the weights of the local characteristic values of thelower body may be increased when the moving direction angle 302 is notwithin a predetermined angle range. The angle ranges mentioned above aremerely examples and are not limited thereto. In Step f91 in thisembodiment, the process follows two different flows; instead there maybe more than three flows in which different detection methods are used.Then, the determining unit 104 links, as illustrated in FIG. 8, theselected matching patterns with the objects for which matching patternsare selected and stores this information in the storage unit 108 (f95).

With the configuration described above, the image processing apparatusaccording to this embodiment can detect a target object using anappropriate matching pattern in accordance with the movement of theobject in an image. Thus, the efficiency of detecting a specific objectin an image is improved.

Other Embodiments

Aspects of the present invention can also be realized by a computer of asystem or apparatus (or devices such as a CPU or MPU) that reads out andexecutes a program recorded on a memory device to perform the functionsof the above-described embodiments, and by a method, the steps of whichare performed by a computer of a system or apparatus by, for example,reading out and executing a program recorded on a memory device toperform the functions of the above-described embodiments. For thispurpose, the program is provided to the computer for example via anetwork or from a recording medium of various types serving as thememory device (e.g., computer-readable medium).

While the present invention has been described with reference toexemplary embodiments, it is to be understood that the invention is notlimited to the disclosed exemplary embodiments. The scope of thefollowing claims is to be accorded the broadest interpretation so as toencompass all such modifications and equivalent structures andfunctions.

This application claims the benefit of Japanese Patent Application No.2010-279889 filed Dec. 15, 2010 and No. 2010-279890 filed Dec. 15, 2010,which are hereby incorporated by reference herein in their entirety.

What is claimed is:
 1. A control apparatus for controlling a detectionprocessing for detecting a specific object, comprising: a memory; aprocessor coupled to the memory which executes the following:determining a moving direction of a moving object detected in a movingimage, wherein the moving direction corresponds to a positionalrelationship between a position of the moving object in a moving imageat a first time and a position of the moving object in the moving imageat a second time; and detecting, if a moving direction of the detectedmoving object is a first moving direction, a specific object bycomparing the detected moving object with a pattern for detecting thespecific object, and detecting, if a moving direction of the detectedmoving object is a second moving direction being different from thefirst direction, the specific object by using at least two of a height,width, a ratio between the height and the width, a size, or a shape ofthe detected moving object.
 2. The control apparatus according to claim1, wherein the processor further executes, in the detecting step, thedetection processing in accordance with detections of a plurality ofmoving objects is prioritized such that the detection processing inaccordance with the detections of the plurality of moving objects isperformed in an order corresponding to each moving direction of theplurality of moving objects determined by the determining step.
 3. Thecontrol apparatus according to claim 1, wherein the processor furtherexecutes, selecting, on the basis of the moving direction of the movingobject detected, a specific object among a plurality of specific objectsand performing the detection processing to detect whether the movingobject having a characteristic is the selected specific object.
 4. Thecontrol apparatus according to claim 1, wherein the processor furtherexecutes performing the detection processing by comparing the movingobject with a matching pattern, the matching pattern indicating a shapeof the specific object or a part of the specific object.
 5. A method ofcontrolling a detection processing for detecting a specific object,comprising the steps of: determining a moving direction of a movingobject detected in a moving image, wherein the moving directioncorresponds to a positional relationship between a position of themoving object in a moving image at a first time and a position of themoving object in the moving image at a second time; and detecting, if amoving direction of the detected moving object is a first movingdirection, a specific object by comparing the detected moving objectwith a pattern for detecting the specific object, and detecting, if amoving direction of the detected moving object is a second movingdirection being different from the first direction, the specific objectby using at least two of a height, width, a ratio between the height andthe width, a size, or a shape of the detected moving object.
 6. Themethod of controlling the detection processing, according to claim 5,wherein, in the detecting step, the detection processing in accordancewith detections of a plurality of moving objects is prioritized suchthat the detection processing in accordance with the detections of theplurality of moving objects is performed in an order corresponding toeach moving direction of the plurality of moving objects determined byin the determining step.
 7. A computer-readable recording medium holdinga program for instructing a computer to perform the steps of:determining a moving direction of a moving object detected in a movingimage, wherein the moving direction corresponds to a positionalrelationship between a position of the moving object in a moving imageat a first time and a position of the moving object in the moving imageat a second time; and detecting, if a moving direction of the detectedmoving object is a first moving direction, a specific object bycomparing the detected moving object with a pattern for detecting thespecific object, and detecting, if a moving direction of the detectedmoving object is a second moving direction being different from thefirst direction, the specific object by using at least two of a height,width, a ratio between the height and the width, a size, or a shape ofthe detected moving object.
 8. The computer-readable recording mediumholding a program according to claim 7, wherein, in the detecting, thedetection processing in accordance with detections of a plurality ofmoving objects is prioritized such that the detection processing inaccordance with the detections of the plurality of moving objects isperformed in an order corresponding to each moving direction of theplurality of moving objects determined by in the determining step. 9.The control apparatus according to claim 1, wherein the detectionprocessing performed varies according to the moving directiondetermined.
 10. The control apparatus according to claim 1, wherein theprocessor further executes: tracking the moving object detected, andwherein the determining step determines the moving direction of thetracked moving object on the basis of a result of the tracking of themoving object.
 11. The method of controlling the detection processing,according to claim 5, wherein, in the detecting step, the detectionprocessing varies according to the moving direction determined in thedetermining step.
 12. The method of controlling the detectionprocessing, according to claim 5, wherein, in the detecting step, thefirst moving object and a second moving object are detected, wherein, inthe detecting step, a first detection processing of detecting whetherthe first moving object is a first specific object having a firstcharacteristic corresponding to a first moving direction of the firstmoving object and a second detection processing of detecting whether thesecond moving object is a second specific object having a secondcharacteristic corresponding to a second moving direction of the secondmoving object are performed.
 13. The computer-readable recording mediumholding a program according to claim 7, wherein, the detectionprocessing varies according to the moving direction determined in thedetermining step.
 14. The computer-readable recording medium holding aprogram according to claim 7, wherein, in the detecting step, the firstmoving object and a second moving object are detected, wherein, in thedetecting step, a first detection processing of detecting whether thefirst moving object is a first specific object having a firstcharacteristic corresponding to a first moving direction of the firstmoving object and a second detection processing of detecting whether thesecond moving object is a second specific object having a secondcharacteristic corresponding to a second moving direction of the secondmoving object are performed.
 15. The control apparatus according toclaim 1, wherein the specific object corresponds to an object having apredetermined size or an object having a predetermined pattern.
 16. Thecontrol apparatus according to claim 1, wherein the specific objectcorresponds to a human body, a human face, an automobile or an animal.17. A control apparatus for controlling a detection processing fordetecting a specific object, comprising: a memory; a processor coupledto the memory which executes the following: determining a movingdirection of a moving object detected in a moving image; in a case wherea plurality of moving objects is detected in the moving image,determining, on the basis of the moving directions of the plurality ofmoving objects determined in the determining step, an order forperforming a determination processing for determining whether at leasttwo of a size, a height, a width, a ratio between the height and thewidth, and a moving speed of each of the plurality of moving objectsmeets a predetermined condition; determining, according to the result ofthe determination processing, whether to perform a matching processingto detect whether each of the plurality of the moving objects is aspecific object; detecting a first specific object, detecting a secondspecific object which is different from the first object, and if a sizeof the detected moving object is smaller than a predetermined size, theprocessor skips detecting the first specific object and the processorexecutes detecting the second specific object, and if the size of thedetected moving object is larger than the predetermined size, theprocessor executes detecting the first specific object and the secondspecific object, wherein the processor does not execute, determiningwhether the moving object which moves to a predetermined direction is aspecific object having predetermined characteristics, a predeterminedsize or a predetermined shape.
 18. The control apparatus according toclaim 17, wherein the processor further selects, when the detectionprocessing for detecting whether a first moving object is the specificobject is performed, a second moving object in accordance with the orderfor performing the detection processing and determines whether thedetection processing for detecting whether the second moving object isthe specific object.
 19. The control apparatus according to claim 17,wherein the specific object corresponds to a human body or a human face.20. A method for controlling a detection processing for detecting aspecific object, comprising: determining a moving direction of a movingobject detected in a moving image; in a case where a plurality of movingobjects is detected in the moving image, determining, on the basis ofthe moving directions of the plurality of moving objects determined inthe determining step, an order for performing a determination processingfor determining whether at least two of a size, a height, a width, aratio between the height and the width, and a moving speed of each ofthe plurality of moving objects meets a predetermined condition;determining, according to the result of the determination processing,whether to perform a matching processing to detect whether each of theplurality of the moving objects is a specific object; executing firstdetection for detecting a first specific object, and executing seconddetection for detecting a second specific object which is different fromthe first object, wherein if a size of the detected moving object issmaller than a predetermined size, the first detection is not executedand the second detection is executed, and if the size of the detectedmoving object is larger than the predetermined size, the first detectionand the second detection are executed.
 21. A computer-readable recordingmedium holding a program for instructing a computer to perform the stepsof: determining a moving direction of a moving object detected in amoving image; in a case where a plurality of moving objects is detectedin the moving image, determining, on the basis of the moving directionsof the plurality of moving objects determined in the determining step,an order for performing a determination processing for determiningwhether at least two of a size, a height, a width, a ratio between theheight and the width, and a moving speed of each of the plurality ofmoving objects meets a predetermined condition; and determining,according to the result of the determination processing, whether toperform a matching processing to detect whether each of the plurality ofthe moving objects is a specific object, wherein in the detecting step,if a moving direction of the detected moving object is vertical on adisplay plane of the moving image, the specific object is detected bycomparing the detected moving object with a pattern for detecting thespecific object, and in the detecting step, if a moving direction of thedetected moving object is horizontal in the display plane, the specificobject is detected by using at least one of the height, the width, theratio between the height and the width, the size, or the shape of thedetected moving object.
 22. The control apparatus according to claim 1,wherein the processor further executes, if a plurality of objects isdetected, determining, based on a number of times a detection processfor the specific object is performed or a number of the specific objectsbeing detected, whether to continue the detection process for thespecific object.
 23. The control apparatus according to claim 1, whereinthe processor skips determining whether the moving object which moves toa predetermined direction is a specific object having predeterminedcharacteristics, a predetermined size or a predetermined shape.
 24. Thecontrol apparatus according to claim 1, wherein the processor detects,if a moving direction of the detected moving object is vertical on adisplay plane of the moving image, the specific object by comparing thedetected moving object with a pattern for detecting the specific object,and the processor detects, if a moving direction of the detected movingobject is horizontal in the display plane, the specific object by usingat least one of the height, the width, the ratio between the height andthe width, the size, or the shape of the detected moving object.
 25. Themethod of controlling the detection processing, according to claim 5,wherein, in the detecting step, if a moving direction of the detectedmoving object is vertical on a display plane of the moving image, thespecific object is detected by comparing the detected moving object witha pattern for detecting the specific object, and in the detecting step,if a moving direction of the detected moving object is horizontal in thedisplay plane, the specific object is detected by using at least one ofthe height, the width, the ratio between the height and the width, thesize, or the shape of the detected moving object.
 26. Thecomputer-readable recording medium holding a program according to claim7, wherein the program further instructs the steps of executing firstdetection for detecting a first specific object, and executing seconddetection for detecting a second specific object which is different fromthe first object, and wherein if a size of the detected moving object issmaller than a predetermined size, the first detection is not executedand the second detection is executed, and if the size of the detectedmoving object is larger than the predetermined size, the first detectionand the second detection are executed.